Pure transformers have shown great potential for vision tasks recently. However, their accuracy in small or medium datasets is not satisfactory. Although some existing methods introduce a CNN as a teacher to guide the training process by distillation, the gap between teacher and student networks would lead to sub-optimal performance. In this work, we propose a new One-shot Vision transformer search framework with Online distillation, namely OVO. OVO samples sub-nets for both teacher and student networks for better distillation results. Benefiting from the online distillation, thousands of subnets in the supernet are well-trained without extra finetuning or retraining. In experiments, OVO-Ti achieves 73.32% top-1 accuracy on ImageNet and 75.2% on CIFAR-100, respectively.

A computational graph in a deep neural network (DNN) denotes a specific data flow diagram (DFD) composed of many tensors and operators. Existing toolkits for visualizing computational graphs are not applicable when the structure is highly complicated and large-scale (e.g., BERT [1]). To address this problem, we propose leveraging a suite of visual simplification techniques, including a cycle-removing method, a module-based edge-pruning algorithm, and an isomorphic subgraph stacking strategy. We design and implement an interactive visualization system that is suitable for computational graphs with up to 10 thousand elements. Experimental results and usage scenarios demonstrate that our tool reduces 60% elements on average and hence enhances the performance for recognizing and diagnosing DNN models. Our contributions are integrated into an open-source DNN visualization toolkit, namely, MindInsight [2].

The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

Image super-resolution is a common task on mobile and IoT devices, where one often needs to upscale and enhance low-resolution images and video frames. While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints. In this Mobile AI challenge, we address this problem and propose the participants to design an efficient quantized image super-resolution solution that can demonstrate a real-time performance on mobile NPUs. The participants were provided with the DIV2K dataset and trained INT8 models to do a high-quality 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU capable of accelerating quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 60 FPS rate when reconstructing Full HD resolution images. A detailed description of all models developed in the challenge is provided in this paper.

视觉变压器在计算机视觉任务中表现出色。但是，其（本地）自我注意机制的计算成本很昂贵。相比之下，CNN具有内置的电感偏置效率更高。最近的作品表明，CNN有望通过学习建筑设计和培训协议来与视觉变形金刚竞争。然而，现有方法要么忽略多层次特征，要么缺乏动态繁荣，从而导致次优性能。在本文中，我们提出了一种名为MCA的新型注意力机制，该机制通过多个内核大小捕获了输入图像的不同模式，并启用具有门控机制的输入自适应权重。根据MCA，我们提出了一个名为Convformer的神经网络。争辩者采用了视觉变压器的一般体系结构，同时用我们提出的MCA代替了（本地）自我注意的机制。广泛的实验结果表明，在各种任务中，应变器优于相似的大小视觉变压器（VIT）和卷积神经网络（CNN）。例如，在ImageNet数据集上，交货式S，Convformer-l实现82.8％的最新性能，top-1的精度为83.6％。此外，在ADE20K上，Convformer-S优于1.5 miOU的Swin-T，在Coco上具有较小型号的Coco上的0.9边界盒AP。代码和型号将可用。

模拟和混合信号（AMS）电路设计仍然依赖于人类设计专业知识。机器学习一直通过用人工智能代替人类的体验来协助电路设计自动化。本文介绍了标签，这是一种从利用文本，自我注意力和图形的布局中学习电路表示的新范式。嵌入网络模型在无手动标签的情况下学习空间信息。我们向AMS电路学习介绍文本嵌入和自我注意的机制。实验结果表明，具有工业罚款技术基准的实例之间的布局距离的能力。通过在案例研究中显示有限数据的其他三个学习任务的转移性，可以验证电路表示的有效性：布局匹配预测，线长度估计和净寄生电容预测。

近年来，破坏预测取得了迅速的进展，尤其是在机器学习（ML）的方法中。理解为什么预测因子使某个预测与未来Tokamak破坏预测指标的预测准确性一样至关重要。大多数破坏预测因素的目的是准确性或跨机能力。但是，如果可以解释中断预测模型，则可以说明为什么某些样品被归类为中断前体。这使我们能够说出传入的破坏类型，并使我们深入了解破坏机制。本文根据J-TEXT上的物理引导特征提取（IDP-PGFE）设计了一种称为可解释的破坏预测变量的破坏预测变量。通过提取物理引导的特征有效地改善了模型的预测性能。需要高性能模型来确保解释结果的有效性。 IDP-PGFE的可解释性研究提供了对J-Text破坏的理解，并且通常与现有的破坏理解一致。 IDP-PGFE已被应用于破坏，因为在J文本上的密度极限实验的密度不断增加。 PGFE的时间演变具有贡献，表明ECRH的应用触发了辐射引起的破坏，从而降低了破坏时的密度。虽然RMP的应用确实提高了J文本中的密度极限。解释性研究指导了RMP不仅会影响MHD不稳定性，而且还会影响辐射轮廓的密度极限破坏的物理机制，从而延迟了密度极限的破坏。

预测不同托卡马克人的破坏是要克服的巨大障碍。未来的Tokamaks在高性能排放时几乎无法忍受中断。很少有高性能的破坏排放几乎无法构成丰富的训练集，这使得当前数据驱动的方法难以获得可接受的结果。能够将在一个Tokamak训练的中断预测模型转移到另一种训练的机器学习方法以解决该问题。关键是一个包含特征提取器的破坏预测模型，该模型能够在Tokamak诊断数据中提取常见的破坏前体痕迹，并具有可转移的破坏分类器。基于上面的问题，该论文首先提出了专门针对Tokamaks上的普通诊断中的破坏前体特征而设计的深融合功能提取器，该特征是根据当前已知的破坏前体，为可转移模型提供了有希望的基础。通过与J-Text上的手动特征提取进行比较，可以证明融合功能提取器。基于在J-TEXT上训练的功能提取器，将中断预测模型转移到East数据中，仅来自East实验的20次放电。该性能与经过1896年出院的模型相当。从其他模型培训方案之间的比较，转移学习表明了其在预测不同托卡马克人的破坏方面的潜力。

从RGB-D图像中对刚性对象的6D姿势估计对于机器人技术中的对象抓握和操纵至关重要。尽管RGB通道和深度（d）通道通常是互补的，分别提供了外观和几何信息，但如何完全从两个跨模式数据中完全受益仍然是非平凡的。从简单而新的观察结果来看，当对象旋转时，其语义标签是姿势不变的，而其关键点偏移方向是姿势的变体。为此，我们提出了So（3）pose，这是一个新的表示学习网络，可以探索SO（3）equivariant和So（3） - 从深度通道中进行姿势估计的特征。 SO（3） - 激素特征有助于学习更独特的表示，以分割来自RGB通道外观相似的对象。 SO（3） - 等级特征与RGB功能通信，以推导（缺失的）几何形状，以检测从深度通道的反射表面的对象的关键点。与大多数现有的姿势估计方法不同，我们的SO（3） - 不仅可以实现RGB和深度渠道之间的信息通信，而且自然会吸收SO（3） - 等级的几何学知识，从深度图像中，导致更好的外观和更好的外观和更好几何表示学习。综合实验表明，我们的方法在三个基准测试中实现了最先进的性能。

图形着色是一个经典且关键的NP硬性问题，是分配尽可能不同颜色的连接节点的问题。但是，我们观察到，最新的GNN在图形着色问题中不太成功。我们从两个角度分析原因。首先，大多数GNN都无法将任务概括为同质性的任务，即在其中分配了不同颜色的图形。其次，GNN受网络深度的界定，使其成为一种本地方法，在最大独立集（MIS）问题中已证明这是非最佳选择的。在本文中，我们专注于流行的GNN类的聚合 - 结合GNNS（AC-GNNS）。我们首先将AC-GNN在着色问题中的功能定义为分配节点不同颜色的能力。该定义与以前的定义不同，该定义是基于同质的假设。我们确定了AC-GNN无法区分的节点对。此外，我们表明任何AC-GNN都是本地着色方法，并且任何局部着色方法都是通过稀疏随机图探索局部方法的极限，从而证明了AC-GNN的非典型性财产。然后，我们证明了模型深度与其着色能力之间的正相关。此外，我们讨论了图形的颜色模棱两可，以应对一些实际约束，例如预固化约束。在上面的讨论之后，我们总结了一系列规则一系列规则，这些规则使GNN颜色均等且功能强大。然后，我们提出了满足这些规则的简单AC-GNN变化。我们从经验上验证了我们的理论发现，并证明我们的简单模型在质量和运行时都大大优于最先进的启发式算法。